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ABSTRACT 

Imbedded  Markov  chains  of  finite  queueing  stystems  with  unit  jumps  at 
regeneration  points  have  an  almost  left  triangular  (in  systems  of  the  type 
G/M/s/N  - in  Kendall  notation  modified  to  include  system  capacity)  or  an 
almost  right  triangular  (in  systems  of  the  type  M/G/l/N)  structure.  Using 
this  structure  a fundamental  recursion  on  the  elements  of  the  transition 
probability  matrix  is  developed,  which  in  turn  helps  derive  first  passage 


1.  INTRODUCTION 


Finite  queueing  systems  have  not  been  studied  so  far  as  extensively 
as  those  with  countably  or  uncountably  infinite  state  space.  The  main 
reasons  for  this  situation  seem  to  be  the  suitability  of  an  infinite 
state  system  approximation  for  a finite  state  system  with  a moderately 
large  state  space  and  the  relative  simplicity  of  a finite  queueing 
system  with  a very  small  state  space.  However,  when  the  state  space 
is  of  medium  size,  in  order  to  provide  usable  results,  the  study  of 
finite  queues  by  themselves  is  necessary.  In  this  paper  we  present 
an  analytic  technique  which  should  prove  significant  in  a wide  variety 
of  such  cases. 


The  class  of  queueing  systems  that  can  be  analyzed  by  the  methods 
developed  in  this  paper  may  be  identified  through  two  subclasses. 

(1)  Multiserver  queueing  systems  with  a general  arrival  process 
(possibly  dependent  on  the  state  of  the  system) , exponential  service 
times  and  a waiting  room  of  finite  capacity  (denoted  as  G/M/s/N). 

(2)  Single  server  queueing  systems,  with  Poisson  arrivals  (pos- 
sibly dependent  on  the  state  of  the  system)  and  a waiting  room  of 
finite  capacity  (denoted  as  M/G/l/N). 

The  procedure  is  based  on  the  imbedded  Markov  chain  of  the  basic 
process  and  therefore  it  is  essential  that  in  the  case  of  multi-server 
systems  the  service  times  be  exponential  and  in  single  server  systems 
when  the  service  times  have  a general  distribution  the  arrival  process 
be  Poisson.  It  is  to  be  noted,  in  queueing  systems  belonging  to  the 
above  classes,  the  transition  probability  matrices  (TPM)  of  the  imbedded 
chains  have  an  almost  triangular  structure  (also  known  as  Hessenberg 
matrices).  If  arrivals  are  one  at  a time  in  the  first  subclass  of 
queues,  the  TPM  is  almost  left  triangular  (a  matrix  in  which  all  the 


elements  above  the  super  diagonal  are  zero)  and  when  the  departures  are 


one  at  a time  in  the  second  subclass  of  systems,  the  TPM  is  almost 
right  triangular  (a  matrix  in  which  all  the  elements  below  the  sub- 
diagonal  are  zero) . 

In  the  next  two  sections  we  consider  an  almost  left  triangular 
transition  probability  matrix  and  an  almost  right  triangular  transition 
probability  matrix  respectively.  Each  section  addresses  to  the  problem 
of  deriving  the  fundamental  recursion,  limiting  distribution  and  first 
passage  times.  Remarks  have  also  been  incorporated  on  the  computational 
feasibility  of  the  results  presented  in  the  paper.  Section  4 on  computational 
procedure  outlines  the  algorithmic  aspects  of  the  model.  The  final  section 
illustrates  this  procedure  with  an  example,  where  a system  of  the  type 
G/M/s/N  described  above  is  analyzed  to  give  most  of  the  measures  of 
performance  of  interest.  Such  systems  arise  in  the  study  of  telecom- 
munications traffic  at  a multiple  channel  node  shared  by  more  than 
one  class  of  customers  (e.g.,  voice  and  data;  see  Fischer  [3];  for 
a complete  analysis  of  the  system  see,  Bhat  and  Raju  [1]). 

2.  AN  ALMOST  LEFT  TRIANGULAR  TPM 

Two  standard  results  from  the  theory  of  finite  Markov  chains  are  given 
below. 

(1)  Let  {0,1,2,...,  N}  be  the  state  space  and  P^ (i, j-0,1,2, . . . N)  be 
the  one  step  transition  probabilities  of  an  aperiodic,  recurrent  and  irreducible 
Markov  chain  {Q^,  n=0,l,2,...  }.  Let  tt®  (Uq  ,tt^  , . . • , be  the  limiting  dis- 

tribution of  the  Markov  chain.  Then  tt  can  be  obtained  by  solving  the  set  of 
simultaneous  linear  equations 

TIP  - TT  (1) 

N 

using  the  normalizing  condition  £ tt.  ■ 1,  where  P is  the  transition  probability 

j-0  J 

matrix  with  elements  P^ (i,j«0,l,2. . . N) . 
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(2)  Consider  the  first  passage  of  the  Markov  chain  from  a state  keB^ 
where  B^i,  1+1,...,  N}  to  any  state  eB.^  where  §^*{0,1,2, . . . , i-1}.  Define 

t£(B.)=  inf{n|QneBl,  given  keB^;  t£  = tJ^).  (2) 

Let  H be  the  substochastic  matrix  obtained  by  partitioning  the  transition 
probability  matrix  P as  follows: 


P00  P01  P0,  i-1  P0i  P0N 


P P P 

iO  il  •”  i,  i-1 


p p p 

NO  N1  N,  i-1 


Then  it  is  known  (see  Kemeny  and  Snell  [4])  that  EtT^B^]  is  given  by  the 
(k-i+l)th  row  sum  of  (I-H)  ^[known  as  the  fundamental  matrix]  and  the  ele- 
ments of  this  row  give  the  mean  number  of  visits  of  the  Markov  chain  to  the 
corresponding  states  before  the  eventual  first  passage. 

Variance  of  first  passage  times  can  also  be  determined  as  functions  of 
elements  of  (I-H)  . 

Given  below  are  results  that  can  be  used  to  simplify  the  matrix  inversion 
procedure  needed  in  determining  the  first  passage  times  and  the  limiting  dis- 
tribution. The  method  exploits  the  almost  triangular  structure  of  the  TPM 
and  is  based  on  recursive  relations  convenient  for  computational  use. 

2.1  The  fundamental  recursion 

Let  the  transition  probability  matrix  P have  an  almost  left  triangular 
structure.  Without  loss  of  generality,  for  notational  convenience  we  shall 
assume  that  the  first  passage  is  from  the  set  of  states  (1,2,...  N}  to  state 
(0>.  Now,  writing  H=H(N),  where  N is  the  order  of  H,  we  have 


N-1,1 


N-1,2 


N-l.N-l 


N-1,N 


-P 


N1 


-P, 


N2 


. -P 


N,N-1 


1-P, 


NN  ' 


For  j>0,  k>0,  define 


Cj> 

c+1 


1_pjj 

“pj+l,j 

_Pj+k-2,j 

"P j+k-1, j 

"Pj+k,j 

“pj,j+l 

1-pj+l,j+l,  "" 

~Pj+k-2, j+1 

"Pj+k-l,j+l 

~P j+k, j+1 

• o 

• • 

“Pj+k-2, j+k-1  1_P j+k-1, j+k-1 
"P j+k-1, j+k 

_Pj+k, j+k-1 
1-P 

j+k, j+k 

and  let  det  (A^])=  Using  the  principal  minors  of  A,P} 


k+1  “k+1 

recurrence  relation 


k+1 


we  have  the 


,(j)  = 

k 


A,'J ' = 0 for  k<0 


*o0)  • 1 


- (1-P  )A^^  - P p a(J) 

k+1  j+k, j+k'  k Pj+k-l, j+k  j+k, j+k-1  k-1  **' 


( 


k-1 


- 1 n p , lpJM 


Jt-0  J+k-l-A,J+k-A J rj+k,ja0  * 

(see,  Faddeev  and  Faddeeva  [2],  p.  31). 

For  computational  purposes  a better  recursive  relation  than  (5)  can  be  derived 
Define 

k+j 


BkJ)  * n p . n 

n-j+l  n 1,n 


8qJ)  = 1 for  all  j>0 


(j>0,k>0) 
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Dividing  (5)  by  £ and  simplifying  we  get 


k+1  P, 


-^1  [%0>  - ■««-« 


where  for  j > 0, 


.0)  = 


(1"pjj)  /pj,j+l 


= 0 for  all  n<0. 
n 


The  recursive  relation  (6)  has  the  following  equivalent  form 


,(j)  _ 


= ifjl  1 a(j+D  _ y rFM3  1 a(j+ 

P k i.  p ak-£ 

L j,j+ij  K ML  j+i,j+i+u 


(j+l+JO 


(j>l,  0<k<N- j ) 


The  equivalence  of  the  two  forms  can  be  established  directly  using  mathe- 


matical induction. 


Because  of  its  usefulness  in  computations  we  shall  identify  equation 


(7)  as  the  fundamental  recursion  and  the  successive  iterates  as  transition 


iterates . The  limiting  distribution  and  the  first  passage  times  for  the 


TPM  can  be  expressed  in  terms  of  these  computationally  attractive  transi- 


tion iterates. 


2.2  Limiting  distribution  of  the  Markov  chain 


In  obtaining  the  limiting  distribution  the  following  inversion  formula 


is  useful. 
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P (~^k+i-l  k+i^  ^ 

k+i-l,k+i  ’*■  1 

a(k+i)  1-P 

p _1 k+i , k+i  = - 

k+i-l,k+i  Pk+i_1>k+i  Pk+i,  k+i+1 


and  in  general  for  any  j (l<j<n-i-l), 


r (k+i 
(k+i)  aj-l 

j ' L Pk+i, 


(k+i+1)  , 

1-1  / k+i+£,k+i 

k+i , k+i+1  £,=0  \ Pk+i+£ , k+i+£+l 


(k+i+l+£)  ~1  _ 

j-l-£  J ~ 


From  the  fundamental  recursion  it  is  obvious  that  the  above  equality  holds. 
This  completes  the  proof  of  the  lemma. 

Based  on  this  lemma  we  have  the  following  theorem. 

Theorem  2.2.1:  Let  (Qn>n=0,l,2, . . . } be  an  aperiodic,  irreducible  finite 

Markov  chain  with  an  almost  left  triangular  transition  probability  matrix 
with  elements  P (i,j=0,l,2, . . . N).  Let  , ttn)  be  the  limiting 

distribution  of  the  Markov  chain.  Then  the  elements  of  tt  are  given  by 


,U+1>/ 

N"j  /PJ,: 


"N  (0<j<N-l) 


where  a^  are  the  transition  iterates  defined  in  (7)  and  we  have  assumed 


PN  N+l=^  t0  facmtate  computation. 


Proof : Rewriting  (1)  we  have 


tt(I-P)  = 0 


where  I is  the  identity  matrix  of  rank  (N+l).  Realizing  that  the  vector  it 
can  be  uniquely  determined  from  the  last  N equations  of  (11)  and  the  nor- 


i 


malizing  contition  £ tt.=1,  we  re-arrange  the  last  N equations  of  (11)  as 
j=0  J 

follows 


-poi 

^ll 

"P12 

~P21 

1-P22  • 

~PN-1,1 

_PN-1,2  1-PN-1,N-1 

-p 

N-1,N 

^PN1,PN2*  " ‘ ’ PN,N-1’  - ^1-PNN^’ 

The  coefficient  matrix  of  the  above  non-homogeneous  system  of  linear  equations 
is  B(N,0)  of  lemma  2.2.1.  Thus  we  have 

‘Vi Vi1  ■ VpwpK—  \s-v  - . 

which  yields 

(1)  P a^  1 

*0  = P [aN-l  ' l N>N_£  N-1-£ 

01  £=0 


* . ^ ra(j+D  _ V jp  a(j+D  i 1<JKN  . 

j PjJ+1  UN-l-j  PN,N-^aN-l-j-£3  1-j-N‘1 

which  can  be  further  simplified  by  using  the  recursion  (6)  and 
assuming  P^  ^^=1  for  notational  convenience.  (It  may  be  easily  verified 
that  this  assumption  does  not  invalidate  the  recursion.)  The  theorem  now 


follows  from  the  normalizing  condition  £ ir  =1. 

j=0  J 


2.3.  (I-H)~  and  the  first  passage  times. 

The  objective  of  this  section  is  to  obtain  the  elements  of  the  inverse 
matrix  (I-H)  using  some  of  the  special  structural  properties  of  the  almost 
left  triangular  TPM.  We  have 
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Lemma  2.3.1: 

detfl-H]  = A<1}.  (lJ 

This  result  is  obvious  from  our  definitions  following  equation  (4). 

Corollary:  Let  C( 1,1)  be  the  co-factor  of  the  (1,1)  element  of  matrix 

[I-H],  Then 

cd.D  = . (1: 

Lemma  2.3.2:  The  co-factor  C(N,1)  of  the  (N,l)  element  of  matrix  [I-H]  is 

given  by 


N-l 

C (N,  1)  = n P 


This  result  follows  if  we  note  that  c(N,l)  is  the  determinant  of  a 


triangular  matrix  of  dimension  (N-l)  with  -P^  ^ (1<J,<N-1)  for  each  of  its 
diagonal  entries. 

For  further  development  we  introduce  the  following  notations.  iet  dT 
be  the  transpose  of  matrix  D.  Define 


X = = [I-H] 


Y i (£>)*  . 

Denote  the  adjoint  matrices  of  X and  Y by 


adj(X)  = ||x, 


(l<k,j<N) 


adJ (Y)  = I |ykjl I (l<k,j<N-l). 


Then  we  have 


Lemma  2.3.3: 


xu  * pi2yi.j-i  <2^:N)  • 


Proof:  We  have  the  set  of  equations 

adj(Xx  )X  = det(X)lr 
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where  adj(X^.)  is  the  first  row  of  adj  (X)  and  1^.  is  the  unit  row  vector 

with  1 in  the  first  position.  But  we  already  know  x,.  from  (13)  and  x,„ 

11  IN 

from  (14).  Therefore  the  remaining  N-2  elements  of  adj^.)  can  be  uniquely 
determined  from  the  last  N-2  equations  of  (17)  by  back  substitution.  Thus 
dropping  the  first  two  equations  of  (17)  we  have  the  linear  homogeneous 
system 


[X12,X13’”  • ’ X1N] 


— 

~P23 

1"P33 

_P34 

_PN-1,3 

-p 

N-1,4  

1-P  -P 

N-l.N-l  N-l ,N 

55 

J 

_p 

rN4  

-P  1-P 

N,N-1  A NN 

= [0,0 

0]. 

(18) 

Similarly  from  the  set  of  equations 
adj  (Yr)Y=det(Y)lr 

(with  definitions  similar  to  those  for  X),  dropping  the  first  equation  we  get 
the  same  system  of  N-2  equations  as  (18)  except  that  the  variable  vector  now 
1S  ’ yl,N-l]‘  Also  the  solution  to  the  set  of  equations  (18)  is 

unique  if  we  know  one  of  the  elements  of  the  variable  vector.  Further  from 
(14)  we  have 

X1N  = P12yl,N-l 

which  relationship  should  then  hold  for  all  elements. 

This  completes  the  proof  of  the  lemma.  The  above  lemmas  lead  us  to  the 


following  theorem. 
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: 


Theorem  2.3.1 


VaU+1)  i 


J-1,U  ' J-lVi  + tn.2P,-l'S  U-Pra  * PB.1>N]  ■ (19 

where  the  product  is  assumed  to  be  1 whenever  the  lower  limt  is  larger  than 
the  upper  limit. 

Proof:  First  note 

N N-2 

j=lXlj  = j^lJ  + X1N-1  + X1N. 

But  from  (14)  we  have 


X!N  ~ JJ/i-l,* 


and  the  last  equation  in  (18)  gives 

(1_PNN)X1N  = PN-1,NX1,N-1 
from  which  we  get 


-i.h-i  ' (1'V  " Vi.r 


Hence 


-l.N-l  + X1N  * (‘-’W,  + "Vw. 


Also  from  (13)  we  have 


X11  = AN-1  * 


The  remaining  terms  can  be  generated  recursively  as  follows  by  repeatedly 
applying  (16)  and  (13). 


I 


[ 


E[T 


11  = ilM*  + + PN-1,N • 


N 


Dividing  the  numerator  and  denominator  by  II  P we  get 

0 = 9 


e[tJ] 


/ 1~PNN~PN-1,N  \ Nr2  AN-j 

l PN-1,N  I jil  J 


1=2 

(J+1 


A- j+1 


' Jl-l.A 


(23) 


/ 


However  from  equation  (6)  and  noting  the  following  definition,  it  follows  that 


A(j+1> 


N 

n p j ! 

£=j+l  £-1»A 


A_/a(J+1)  - Y^P  A(J+1>  \ 

,j+1  ( £=0  N.N-A-l-j-H J 


(24) 


and 


,(D 


N-l 


. a(l)  _ "y"p  (1) 

N N-l  * 


(25) 


^ P£-l  l 


Now,  substituting  from  (24)  and  (25)  in  (23)  we  get  the  required  expression  (21), 
In  the  recursive  relation  (6)  assuming  k=N-l  (note  that  the  recursion  re- 
stricts k to  be  less  than  N-l)  we  get 

„(1) 


_1 [>  - Yr  a(1> 

V(H-iLN'1  jio  x.B-rt-i-nJ  • 


(26) 


Since  the  Markov  chain  is  assumed  to  have  N states  P„  does  not  exist. 

N,N+1 


However,  if  we  assume  P^  N+^  = 1,  the  transition  iterates  aj^  (0<j<N-l)  remain 


,(D 


unchanged  and  thus  a^  in  this  case  actually  represents  the  term  within  the 


braces  of  the  right  hand  side  of  equation  (26).  Similarly,  we  can  show  that 


.«+«  = _i ^ (j+D  _ Y"JP  a (J+1)  1 

N"J  PN , N+l  L N_1“J  £“0  PN,N-£aN-l-j-£  J 


(27) 


Now,  substituting  (26)  and  (27)  in  (23)  and  making  a change  of  variable  we 
obtain  the  required  expression  (22).  This  completes  the  proof  of  the  theorem. 


m 
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It  may  be  recalled  that.  Initially  without  loss  of  generality  in  the 
procedure*  we  considered  the  first  passage  from  state  1 to  state  0.  On  the 

other  hand,  if  the  taboo  states  had  been  {0,1,2. . .1-1}  and  the  first  pas- 
sage considered  was  from  state  i to  any  one  of  these  states  , the  following 
expressions  would  have  resulted  instead  of  (21)  and  (22) . 


e[t£(b±)] 


f , 1+J)  y-j,  tut)  ) 

Vi,«  J sii  p1+j-i,i+j  y-1-!  «io  s-8-' 


,u>  _ j 

Vi  y 


N-i 


(D 


u p a'  ' 

«.=0  N,N-&  N-i-5. 


(28) 


or 


N-i  . 

1 + 1 


j=l  ri+j-l,i+j 


.(i) 


N-i+1 


(29) 


with  P 


N,N+1 


So  far  we  have  concentrated  on  the  first  row  of  the  matrix  (I-H)”1  yield 


ing  the  mean  first  passage  time  from  state  1 to  state  zero.  To  determine 

.-1 


first  passage  times  from  other  states  all  elements  of  (I-H)"1  need  to  be 
known.  They  are  given  in  the  following  Lemma. 


Lemma  2.3.4 


Let  adj[I-H]  be  denoted  by 


X=  II 


Jk 


(j»k=l,2,3, . . . N), 


Then 


/n  \ 

HP,,. 


(kfl)  (1)  _ (1)  (k+1) 

Vk  aj-l  aN  1-k-l 


k , k+1 


(30) 


Proof : For  the  sake  of  conciseness  only  essential  steps  of  the  proof  will 

be  given. 

Clearly 


[I-H] . X.k  = det[I-H] .1 


(31) 


(l<k<N. ) 


^rom  Theorem  2.3.1,  xlk  (l<k<N)  are  known.  Hence  for  any  k,  the  remaining 
N-l  elements  of  X.^  can  be  uniquely  determined  from  any  N-l  equations  of  the 
independent  non-homogenous  system  of  equation  (31).  Dropping  the  last  equa- 
tion of  (31)  and  re-arranging  we  have  for  k = 1,2..., N-l 


B(N-1 , 1) 

X2K 

X3K 

- det(I-H)I.k  - 

1 

21 

• 

*NK 

where  B(n,k)  is  as  defined  in  Lemma  2.2.1,  and  I is  a (N-l)x(N-l)  identity 
matrix.  Premultiplying  both  sides  by  B ^"(N-  , ),  using  the  re- 

cursion (6)  as  a simplifying  feature  and  the  form  of  the  inverse  of  8(N-i,i) 
we  get  for  (k*=l,2, . . . ,N-1) 


X2k 

X3k 

• 

ss  X 

ka,l 

det(I-H] 

0 

• 

• 

0 

• 

X 

NK 

lk 

* 

:u> 

N-l 

Pk,k+1 

1 

a (k+1) 
al 

’(k+1) 

N-l-k  0 


A similar  operation  for  k=N  gives 


X2N 

X3N 

= x. 

a(1) 

a2 

• 

in 

• 

*NN 

*(i) 

N-l 

The  lemma  now  follows  by  substituting  known  expressions  for  det[I-H]  and 
x (k=l,2...  N). 
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Corollary 

(a)  The  elements  of  the  matrix  (I-H) 
are  given  by 


-1 


'jk 


(k+1)  (1) 
N-k  a1-l 

• aN 


- a 


(k+1) 

j-k-l 


II  V P 


k,k+l 


(j,k«l,2,...  N) 


(34) 


(b)  If  the  first  passage  considered  is  from  a state  eB^  * {i,i+l,...  N) 
to  a state  eB^  = {0,1,2, .. .i-1},  we  would  have  considered  an  H matrix  with 
dimension  N-i+1.  Then,  let  [I-H(N-i+l)]  * 

The  elements v^(N-i+l)  are  obtained  as 


V<N-i+l) 


(j ,k*l,2. • . , N-i+1). 


v (N-i+1) 


(i+k)  (i) 

N-i+l-k  1-1  (i+k) 

^ aj-k-l 


a.(i) 
^J-i+1 


j 

j?  i+k-1, 


i+k 


The  proof  of  this  result  follows  exactly  the  same  lines  as  that  of  (34). 


Theorem  2.3.3:  (a)  Let  EfT^XB.^)]  be  the  first  passage  time  of  the  Markov 

chain  to  state  0,  having  initially  started  from  state  k.  Then 


k-1  a 


(j+1) 


EIT^)]  = E[Tj]a£>  - l 


j=l  j,J+l 


(k=2, 3, . . . N) 


(35) 


where 


e[tJ] 


(j+1) 


N-l  a'-. 

1+  I 


A1’- 


j-i  j.j+i. 

^ (b)  Let  ^[^(B^)]  be  the  mean  first  passage  time  of  the  Markov 

chain  to  any  of  the  states  eB  = {0,1,2. . .i-1}  from  state  kcB  * {i,i+l,...,  N}. 


Then 


EfTk<B±)1 


*kX  E[^(Bt)) 


k-i  a 


(i+m) 

k-i-m 


where 


E[tJ(B1) ] 


- I 

m*l  i+m- 1, i+m 

N-i  a*1*"} 

^ + j*  N-i+l-m 


/ 


(i) 


/ N-i+1  ‘ 


m=l  i+m-1, i+m 

Part  (a)  of  the  theorem  follows  from  direct  substitution.  Part  (b)  can  be 
derived  by  similar  methods. 


2.4  Remarks  on  computational  feasibility 


A comparison  of  expressions  for  the  limiting  distribution  ir  and  the  first 
passage  time  E[lJ]  indicates  that,  if  we  compute  it,  we  have  all  the  informa- 
tion needed  to  obtain  the  latter.  To  determine  the  steady  state  probability 

vector  tf,  we  need  only  N transition  iterates  a^^\  (j=0,l, . . . ,N-1) . 

(N) 

Noting  that  a'  = (l-P.^J/P.,  where  P„  „,«1  the  remaining  N-l  transition 

NN  N } Nt  1 N ) Nt  1 

iterates  can  be  generated  successively  using  the  fundamental  recursion  (7), 

and  starting  with  k=l,j=N-k  and  successively  incrementing  k up  to  N-l. 

Given  the  matrix  P,  it  can  be  deduced  that  the  generation  of  N successive 

transition  iterates  would  require  a total  n^+^~ operations  (N^-  multi- 

N (N+l) 

plications/divisions  and  — ^ — ~ ~ subtractions/additions)  on  a digital 
computer. 

Many  times,  systems  such  as  G/M/s/N  result  in  a transition  probability 
matrix  with  last  two  rows  identical.  Modified  expressions  for  mean  first 
passage  times  and  the  limiting  distribution  can  be  given  for  this  case  much 
the  same  way  as  the  general  case.  However,  the  resulting  expressions  are 
computationally  advantageous  only  in  systems  with  small  N(N<6)  or  in  systems 
such  as  G/M/l/N  in  which  the  recursions  (6)  and  (7)  turn  out  to  be  the  same. 
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3.  AN  ALMOST  RIGHT  TRIANGULAR  TPM 

The  procedure  for  analysis  of  an  almost  right  triangular  TPM  would 
follow  easily  if  we  note  the  following. 


For  a matrix 


define  its  reverse  to  be 


= (x 

2 $ *2  * * • • • » 

x , , x ) , define 
n— I n 

its 

(v 

Xn-1*  x3* 

x2,  x1>. 

yll 

y12 

yi,n-l 

yln 

y21 

y22 

y2,n-l 

y2n 

yn-l 

,1  Vl,2 

yn-l,n-l 

yn-: 

ynl 

yn2 

yn,n-l 

yn,i 

to  be 

ynn 

yn,n-l 

yn2 

ynl 

yn-l 

y 

,n  n-l,n-l 

yn-l,2 

yn-: 

c 

CSI 

V2,n-1 

y22 

y21 

yln 

yl,n-l 

yl2 

yll 

Then, 

* * * 

i)  XY  - X and  X Y = X represent  identical  sets  of  linear  equations 

ii)  If  Y is  ALT,  then  Y*  is  ART. 

iii)  (Y  *.e)  =•  (Y  ) *.e,  where  e is  a vector  of  l's. 

Therefore,  all  results  for  the  ALT  TPM  in  Section  2 are  applicable  to  the 
ART  TPM  if  the  order  of  the  sequence,  {0,1,2,  ,N-1,N}  is  reversed.  There- 
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fore,  In  this  section  we  shall  concentrate  on  just  giving  the  major  results 
without  details. 


3,1  THE  FUNDAMENTAL  RECURSION 

Following  equations  (3),  (4)  and  (5)  of  Section  2,1,  instead  of 
recursive  relations  (6)  and  (7),  for  j>0  and  k>0,  we  have 


4j) 


a«>  - Cl-Ww 


,(j) 


a""  * 0 for  all  n<0* 
n 


k+1 


j+k+l,j+k 


'■(1_P)+k,i+kl 


l v-M+k  ■£* 1 (36) 


or  alternately. 


Pj.,l+A 

pj+ii+i,j+ji 


(j+Jt+D 

ak  -£ 


(37) 


In  this  case  we  shall  identify  (36)  as  the  fundamental  recursion.  As  before 
we  shall  consider  a£^  's  as  transition  iterates. 


3,2  LIMITING  DISTRIBUTION  OF  THE  MARKOV  CHAIN 

Theorem  3,2,1.  Let  {Q  ,n»0,l,2, . . . } be  an  aperiodic,  irreducible  finite 
Markov  chain  with  an  almost  right  triangular  transition  probability  matrix 
with  elements  P (i,  j-0,1,2. . . N)  . Let  tt  = (tTq.t^,  . . . .ir^)  be  the  limiting 
distribution  of  the  Markov  chain.  (a)  Then  the  elements  of  ir  are  given  by 


(38) 


IT" 


cr : — 


-20- 


« 


(b)  Further,  let  the  first  two  rows  of  the  TPM  be  identical  (i.e.,  = P 

( j=0,l,2, . . . , N)).  Then,  the  elements  of  ir  are  given  by 

-1 


*0  ~ 


W1  = IP 


aN  1 N"2 

(4U 

42>] 

p + 1 

L P10  k=0 

\Pio 

P21  / 

- 1 : ir 


0 

gX)  aW\ 

a<  ’ ai-2  1 


\ IQ 
/aO 


P 


j=2,3,...,  N. 


(39) 
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Proof  Consider  the  linear  homogenous  system  of  equations 
ir(I-P)  = 0. 

The  elements  of  ir  are  determined  much  the  same  way  as  in  Theorem  2.2.1  of 
Section  2.  Here  we  re-arrange  the  first  N equations  to  express  ir^  (j=l,2,...  N) 
in  terms  of  tt^  and  use  the  recursion  (37)  in  simplifications.  Part  (b)  of  the 
theorem  utilizes  the  special  structure  of  the  first  two  rows  of  the  transition 
probability  matrix  P. 

3.3  (I-H)  ^ and  first  passage  times. 

Analogous  to  lemma  2.3.4,  we  have 
Lemma  3.3.1 

r 

P 


*k. 


m 


m-1 


■ a(j+k) 
Pj+k, j+k-1  m_k_1 


(k=l, 2, . . , N-l,  m=l,2, , . . *N) 


Then 


1 


N-k-1 


Sk  = SN  " P 


, (j+k) 


N P 


j+k » j+k- 1 m=0 

[l+Ya(j) 

I » j-1  m*l 


(k=l,2, . . . ,N-1) 


Proof  follows  by  direct  computation. 

These  results  lead  us  to  the  mean  first  passage  times.  We  have 


Theorem  3.3.1.  Let  {Q^,w-0,1,2, . . . } be  an  aperiodic,  irreducible,  finite 
Markov  chain  with  an  almost  right  triangular  transition  probability  matrix 


(i,j 

• 

O 

II 

• • »N) . 

Let  1 

1 in 

terms  of 

number 

of  'Ll 

a(1) 

aN-l 

N-2  / 
+ T / 

m 

a(2) 

m 

P10 

L I 
m=0  l 

1 

o 

rH 

Oi 

P21  • 

Proof : We  have 

N _i 

E[T^]  = First  row  sum  of  Cl-H ) . 

Clearly  the  matrix  [I-H]  is  the  same  as  X with  j replaced  by  1.  Hence  we  have 

'H-h-h  T 42> 

r21  m= 0 m 


where 


. N-l 

3N  = T~  1 + l 
10  m=»l 


The  result  follows  after  simplification  using  transition  iterates. 

N 

Similar  results  follow  for  E[T^(B^)]  when  the  system  initially  starts  from 
state  k. 

Suppose  we  consider  the  first  passage  of  the  Markov  chain  from  state  i(>l) 

to  the  set  of  states  {0,1,2, ...  ,i-l}.  Then  for  [I-H(N-i+l) ]-1  * II  x,  II  similar 

11  km  1 

methods  would  yield  the  result 


km  P 


i,i-l 


am-l  - am-k-i  ! ( 1< k<N- i , 1<  m<N-i+l ) 

i+k, i+k- 1 m K 1 ' 


XN-i+l,m=xN-i,m  5 XN-i+l,N-i+l  “ Vi/. 


Sk  ’ SN-i+l  " P 


, N-i-k 

— - — I 

i+k, i+k-1  m=0 


(i+k) 
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where 


N-i+1 


i,i-l 


e[t.(b1)] 


N-i 
1 + 1 
m=l 


N-i-1  /a(i) 


i,i-l 


.(i+1) 


\ p p 

'fi »i-l  i+l,i 


3.4  Remarks  on  computational  feasibility 

It  may  be  noted  that  in  comparison  with  the  almost  left  triangular  ma- 
trices, more  computational  effort  is  needed  in  the  case  of  almost  right  tri- 
angular matrices.  As  can  be  seen  from  equation  (38)  N transition  iterates 
are  required  to  obtain  the  limiting  distribution  and  as  is  evident  from 

N 

equation  (43)  a total  of  2(N-i)-l  iterates  are  required  to  determine  E[T\  (B^)]. 

2 2 

Thus  in  general,  the  generation  of  these  iterates  would  involve  N + (N-i)  + 

2 

(N-i-1)  divisions/multiplications  and  N(N+1)  (N-i  ) (N-i  fl)  (N-i-l)(N-i) 

2 2 2 

additions/subtractions.  However  if  the  first  two  rows  of  the  TPM  are  iden- 

N 

tical  it  is  seen  from  equation  (39)  that  the  vector  it  and  E[T^]  can  be 

determined  from  the  same  set  of  2(N-1)-1  transition  iterates.  Moreover,  for 

N 

any  i (0<i <N) , the  determination  of  tt  together  with  E[T.  (B.)]  would  also  require 
less  computational  effort  if  we  take  into  account  the  identical  nature  of  the 
first  two  rows  of  the  transition  probability  matrix. 


J - .*■  i 


4.  THE  COMPUTAT IONAL  PROCEDURE . 


In  this  section  we  present  the  computational  procedure  to  determine  the 
steady  state  distribution  and  the  mean  first  passage  times  discussed  earlier, 
in  an  algorithmic  form  suitable  for  computer  implementation.  Let 

= {i,i+l,,..,  N}  and  for  the  sake  of  simplicity  we  shall  denote  the 

N N 

first  passage  time  (B^)  by  T^. 


Case  i:  P is  an  almost  left-triangular  matrix. 

Initialize:  Given  0 < e < 1;  set  a^  = e for  (1  < j < M + 1) . 

[Remark: 

Note  that  the  particular  value  of  e is  very  much  dependent  upon  the 
magnitude  of  N and  the  computer.  For  small  to  medium-range  values 
of  N,  taking  e = 1 may  save  some  valuable  computer  time. 

For  large  values  of  N,  however,  serious  overflow  problems  may  result. 
This  can  be  brought  under  control  to  some  extent  by  suitable  scaling 
a£j)  for  j>0. ] 


Step  1:  Compute  vector  it. 

„ _ (N)  (N-l)  (N-2) 

Generate  a^  , a^  , 


, a^^  in  that  order 


using  the  fundamental  recursion  (7).  Note  that  these  N 
iterates  car.  be  successively  generated  by  setting  k=0, 
j=N-k  and  successively  incrementing  k up  to  N-l. 


Set 


'1 


-(i)  (w*, .,«) 


(0  < j < N - 1) 


Normalization: 


Step 

[Remark: 


Set  tt  = [1+  l it , ]- 
j=0  3 

and  it  » F * TT  (0  < j < N - 1) 

j J N - 

Go  to  Step  2 or  4. 

2:  Compute  the  expected  values  of  the  function  T^  (i  < k < N) . 

N rh 

Note  that  E[T^]  is  given  by  the  (k-i+1)  component  of  the  column 
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vector  of  row  sums  of  the  fundamental  matrix.  If  we  only  need 
N 

E[T^]  (0  < i < N)  , we  can  obtain  this  from  the  above  N iterates 
using  the  following  expression, 


E[T^] 


.eV 


1 


+ 


N-i 


l 

j=l 


a(1+j) 

N-i+l-j 

Pi+j-l,i+j 


(45) 


Otherwise,  in  addition  to  these  N iterates,  we  need  to  generate 

12  M 

-p(N-i)  iterates  to  compute  E[Tf_]  for  all  k e B.,] 

^ K 1 

N 

Obtain  the  value  of  E[T\]  using  equation  (45).  For  any 


fixed  k > i,  generate  a^  , 
that  order  using  recursion  (7) 


(k-1)  (k-2)  (k-3) 

» 33 


Ci)  . 

• Vl ln 


E[T»]  - E[ljl  a<»  - i j 


k-i  a 


(i+j) 

‘k-i-j 


j=l  i+j -1, i+j 


Go  to  Step  4. 
Step  4:  Stop. 


Case  ii:  P is  an  almost  right  triangular  matrix. 

Initialize:  Given  0 < £ < 1;  set  a^^  = £ for  (0  < j < N) 

Step  1:  Compute  vector  tt. 

If  the  first  two  rows  of  P are  identical  and  if  one  is 

N 

interested  in  obtaining  E[T^j  subsequently  then  go  to 
Minimize. 

Generate  a^^  (1  < k < N)  successively  using  recursion  (36) 

Set  TTj  = a j0)  (1  < j < N) 

Go  to  Normalize, 


Generate  a 


(t) 

k 


(1  < k < N - t)  for  t=l, 


2 using  (36) 


Minimize : 


Set  ir^  «■ 


(1)  .(2) 


V (*)*?-£*  <2^s) 


Normalize: 


Set  it. 


N -1 

1 + I 

j=l  J 


Wj  * "j  * *0  (1  t i i N) 

• Go  to  btep  2 or  4. 

Step  2:  Compute  the  expected  values  of  the  function  (i  < k < N) 

1 2 XT 

We  need  to  generate  ^(N-i)  iterates  to  compute  F[t£]  for  all  k n 
Generate  a^'^  (1  < m < N - i)  successively  using  (36) 


E[T  ] = i 


For  any  fixed  k e B, , generate  a^1^  (l<m<N-k-l) 
i m 

using  (36) 


E[Tk]  " EfTN}  Hi 


Go  to  step  4. 


N-k-1 


k+1 , k / m=0 


(k+1) 


Step  4:  Stop, 


5.  THE  QUEUE  G/M/s/N  - AN  EXAMPLE 


Consider  an  s-channel  queueing  system  with  two  classes  of  customers 

with  the  same  service  rate  and  exponential  service  time  distribution. 

Let  t, , t„,  ...  be  the  arrival  epochs  in  the  system,  and  Z =t  -t  , be 
l £■  n n n-1 

independent  and  identically  distributed  random  variables  with 

Ptzn  - X1  = Mx)  (x  t 0) 

and  E[ZnJ  = A \ n=l,2,3...  . When  an  arrival  occurs,  the  probability  that 

it  is  a class  1 arrival  is  p and  the  probability  that  it  is  a class  2 arrival 
is  1-p. 

The  customers  are  served  by  s service  channels  in  parallel. 

All  services  are  assumed  to  be  exponential  with  the  same  mean  y ^ . The 
buffer  capacity  is  such  that  N(>s)  customers  are  allowed  in  the  system  at 
any  time.  Class  1 customers  have  a higher  priority  in  gaining  access  to  the 
system.  If  the  number  of  customers  in  the  system  ia  >k  (k<s),  an  arriving 
class  2 customer  is  denied  access  to  any  service  channel,  and  immediately  he 
leaves  the  system  without  service.  But,  a class  1 customer  is  denied  access 
to  the  system  only  when  the  number  of  customers  In  the  system  is  N (no  buffer 
space  available).  Class  1 customers  waiting  in  the  buffer  are  served  on  a 
'first  in  first  out'  basis.  Let  Q(t)  be  the  number  of  customers  in  the 
system  at  time  t and  be  the  number  of  customers  in  the  system  just  before 
the  nth  arrival  epoch.  (Note  that  the  incoming  customer  is  not  included  in 
Qn.)  Becuase  of  the  exponential  nature  of  service  times,  it  is  well  known  that 
{Qn>  n-1, 2, 3,...  } is  a Markov  chain  imbedded  in  the  stochastic  process 
{Q(t),  t>0}.  A Markov  chain  is  completely  specified  by  its  transition  prob- 
ability matrix.  Let 
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’ij  * pfQn+1  * J I Qn  * i]  (i.j  - 0,1,2,...,  N). 


We  have 


« ■ <?>  i < k,  j < i + 1 (45a) 

■ PC1;1)  f° (l-e-lJX)  i+1_^e-^lJxdA(x) 

J 0 

+ (l-p)(|)  r (l-e~UX)i~je"juxdA(x)  k<i<8,  j <1+1  (45b) 

J 0 “ 


—syx  (syx)  - A/  . 
P J (i+l-j) ! dA(x) 


i+l-j 


00  —syx  (syx) 


+ (1-p)  r e 
0 


dA(x)  1 < s - 1,  1 + 1>  J > s (45c) 


-p  r r *-svt  suc')u-^(*-t>is-ie-j>'(’-')dtd»u) 

x=*0  t=*0  v J 

+ (i-p)  r r «-sut 

x=0  t=0  u ■L  -)  ‘ J 


e~jy  (:<-t)  dtdA(x)  i > s,  J < s. 


rg'syx-^j  dA(x)  1 = N,  N > j > 


r r e-8^  w!-1-8  8)[1  -y(x-t),s-j 

x-0  t-0  (N-l-s) ! sy(^)[l  e J 


e-jy(x-t)  dtdA(x)  i - n,  j < s . 


For  computational  purposes  the  above  expressions  can  be  simplified  by  ex- 
pressing them  in  terms  of  the  following  transform: 

Yj(6)  - / e 6x  dA(x)  (j  - 0,1,2,...  ) 


(45d) 


(45e) 


(45f) 
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Then  we  get 


Pij  " f (-Dr  (1'J+1)  YQ((r+j)M)  i < k,  i-j+1 


r=0 


PC1;1)  Y 1 C-l)r  (1_J+1)  y ((r+j)p) 

J r=0  u 


i-J 


+ (1-p)  (h  l (-l)r  y ( (r+j )y)  k < i < 8,  j < i 

J r=0  u 


P Yi-j+l  (sM)  + C1_P)  Y±_j  (sM)  i < s - 1,  i + 1 > j 


! 

i 
I 

B 


" P <J>  j a (Sr-J)  (^)il"1+S'1]  Y,(sy) 

J A*=i-s+l  r=0  s * 

00  s-j  £— i+s 

+ (1-P)  (j)  I l l (S-J)(5ZlZl)  ] y (sy)  i > s> 
J A=i-s  r*0  r s * “ 


YN_j(sP)  i=N,  N>j>s 


sr  S“J  _ e . . A-W-s 

Cj)  l Cl  (-1)  C ~j)  (Srj~r)  ] Y£(sU)  i - N 

J A=N-s  r=0  r s A 


Measures  of  performance 

p : Offered  load  per  channel  (=  X/sy) 

* 

p : Effective  offered  load  per  channel 

(channel  utilization) 


p [axp  + a2(l-p) ] 


N 


where  a,  * l-ir„;  a_  * 1 - 7 it 

1 N 2 r 

r**k 

E(Q)  : Mean  number  in  system  at  arrival  epochs. 

V(Q)  : Variance  of  number  in  system  at  arrival  epochs. 
E(W)  : Mean  delay  for  class  1 customers 


(47a) 

. + 1 (47b) 

± B (47c) 

j < s (47d) 

, j < s.  (47f) 
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I 

i 


' <>+»  Vj  • 

PB^  : Probability  of  blocking  for  class  i (1*1,2) 

N 

PB1  " p1TN  5 PB2  = (1_p)  I 

r=k 

SA  : System  adequacy  (system  response  to  offered  load) 

* 

* p /p 

AR^  : Access  ratio  for  class  1 

al 


E(BC)  : Mean  busy  cycle 

= J [1  + Y0(y)  Eft”  (B1)]I. 
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